Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add linode_database_postgresql_v2 resource #1680

Merged
merged 18 commits into from
Jan 13, 2025

Conversation

lgarber-akamai
Copy link
Contributor

@lgarber-akamai lgarber-akamai commented Nov 27, 2024

📝 Description

This pull request implements a new linode_database_postgresql_v2 resource supporting the new Aiven-backed managed database clusters.

Design Caveats

Flattened Fork Object

The fork block has been flattened into root-level fork_source and fork_restore_time attributes because:

  1. HashiCorp recommends against making use of blocks in new (non-migrated) Framework resources.
    a. Additionally, treating it as a block would have inconsistent usage with the updates object.
  2. When implemented as an ObjectAttribute, all sub-fields must be specified. Because restore_time is treated as optional and computed as an API, there is a valid case where the user does not want to specify it explicitly but does want to access the computed restore time.

Independent Pending Updates Set

The updates.pending set has been split into a root-level pending_updates set, which allows users to specify their updates configuration as a single object.

Flattened Hosts and Credentials Blocks

The hosts and credentials blocks have been flattened into root-level fields to simplify our implementation and to make unknown/null behavior a bit more predictable.

✔️ How to Test

The following test steps assume you have pulled down this PR locally and do not have your account configured for legacy DBaaS clusters.

Unit Testing

make unit-test

Integration Testing

make PKG_NAME=linode/databasepostgresqlv2 int-test

Manual Testing

  1. In a terraform-provider-linode sandbox environment (e.g. dx-devenv), apply the following configuration:
 resource "linode_database_postgresql_v2" "test" {
  region = "us-mia"
  label = "tf-test"
  type = "g6-nanode-1"
  engine_id = "postgresql/16"

  allow_list = ["10.0.0.2/32", "10.0.0.3/32"]
  cluster_size = 3

  updates = {
    hour_of_day = 2
    day_of_week = 3
    frequency = "weekly"
    duration = 4
  }
}

NOTE: Database creations should take ~15 minutes.

  1. Ensure the apply was successful and the database has been properly created in Cloud Manager.
  2. Re-apply the configuration and ensure no changes are proposed.
  3. Make arbitrary changes to the allow_list, cluster_size, type, and updates attributes.
  4. Re-apply the configuration.
  5. Ensure the apply was successful and the database has been properly updated in Cloud Manager.
  6. Re-apply the configuration and ensure no changes are proposed.
  7. Append the following to the end of your configuration to create a new database forked from the first database:
resource "linode_database_postgresql_v2" "forked" {
  region = "us-mia"
  label = "tf-test-forked"
  type = "g6-nanode-1"
  engine_id = "postgresql/16"

  source = linode_database_postgresql_v2.test.id
}
  1. Re-apply the configuration and validate that a new forked database has been created in Cloud Manager.
  2. Re-apply the configuration and ensure no changes are proposed.
  3. Destroy the configuration and ensure all databases are deleted as expected.

@lgarber-akamai lgarber-akamai added the new-feature for new features in the changelog. label Nov 27, 2024
@lgarber-akamai lgarber-akamai force-pushed the new/postgresql-v2 branch 2 times, most recently from 7b9f0ee to b90bf13 Compare December 2, 2024 22:02
Comment on lines +203 to +229
db, err := client.GetPostgresDatabase(ctx, id)
if err != nil {
if lerr, ok := err.(*linodego.Error); ok && lerr.Code == 404 {
resp.Diagnostics.AddWarning(
"Database no longer exists",
fmt.Sprintf(
"Removing PostgreSQL database with ID %v from state because it no longer exists",
id,
),
)
resp.State.RemoveResource(ctx)
return
}
resp.Diagnostics.AddError(
"Unable to refresh the Database",
err.Error(),
)
return
}

resp.Diagnostics.Append(data.Refresh(ctx, client, db.ID, false)...)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's probably a good way to eliminate the redundant network request here but I can't think of one. If anyone has any ideas please let me know!

@@ -0,0 +1,407 @@
//go:build integration || databasepostgresqlv2
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests can take up to 30 minutes to run due to the need for databases to be updated, resized, forked, etc.

We should discuss ways to reduce the impact this has on our test suite runtime.

go.mod Outdated
@@ -125,3 +123,5 @@ require (
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)

replace github.com/linode/linodego => github.com/ezilber-akamai/linodego v0.0.0-20241202200116-8acd7eb1ae45
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Remove this before merging

@@ -0,0 +1,41 @@
package unit
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we already have a package for unit test utils, and if not is there a better place for this package to live?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be fine to leave this here for now. It will be good to look through the repository at some point to evaluate unit test coverage and possible improvements in a separate ticket.

@@ -50,7 +50,7 @@ jobs:
echo "LINODE_TOKEN=${{ secrets.LINODE_TOKEN_USER_2 }}" >> $GITHUB_ENV
;;
"USER_3")
echo "TEST_TAGS=instanceconfig,instancedisk,instanceip,networkingip,objcluster,objkey,profile,rdns,region,regions,stackscript,stackscripts" >> $GITHUB_ENV
echo "TEST_TAGS=databasepostgresqlv2,instanceconfig,instancedisk,instanceip,networkingip,objcluster,objkey,profile,rdns,region,regions,stackscript,stackscripts" >> $GITHUB_ENV
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a pretty arbitrary decision, let me know if there's a better matrix user to run this under 🙂

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good decision since USER_3 takes the least time to finish usually. In future, as test coverage grows for database v2, maybe we can separate and dedicate a new test account

@lgarber-akamai lgarber-akamai marked this pull request as ready for review December 5, 2024 16:05
@lgarber-akamai lgarber-akamai requested a review from a team as a code owner December 5, 2024 16:05
@lgarber-akamai lgarber-akamai requested review from jriddle-linode and ykim-akamai and removed request for a team December 5, 2024 16:05
@lgarber-akamai lgarber-akamai added the do-not-merge PRs that should not be merged until the commented issue is resolved label Dec 5, 2024
@lgarber-akamai lgarber-akamai removed the do-not-merge PRs that should not be merged until the commented issue is resolved label Jan 13, 2025
go 1.22.0

toolchain go1.22.5
go 1.23
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is necessary to use the iterator helpers introduced in this PR

@ykim-akamai
Copy link
Contributor

LGTM, manual steps and tests all pass consistently. Great work!

@ezilber-akamai ezilber-akamai self-requested a review January 13, 2025 19:19
Copy link
Contributor

@ezilber-akamai ezilber-akamai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Integration and manual tests work great locally. Nice work!

@lgarber-akamai lgarber-akamai merged commit 7aeb26e into linode:dev Jan 13, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new-feature for new features in the changelog.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants